forked from thanos-io/thanos
-
Notifications
You must be signed in to change notification settings - Fork 12
cherry pick upstream commits #284
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
yuchen-db
wants to merge
11
commits into
db_main
Choose a base branch
from
yuchen-db/upstream-fix
base: db_main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Creation of the index header shares the underlying storage so we should use singleflight here to only create it once. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> (cherry picked from commit 4b31bba)
Signed-off-by: 🌲 Harry 🌊 John 🏔 <johrry@amazon.com> (cherry picked from commit ba255aa)
Protect reads and writes with the mutex. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> (cherry picked from commit ebd746c)
I don't see any reason why this must not be blocking - on boot we load up all tenants and open up their TSDBs. This is blocking. When receiving a request, a spin up a new goroutine (currently) to open the TSDB. By definition, that new TSDB will be completely empty because we already load pre-existing TSDBs on boot. Init the TSDB only once with a singleflight group. Fixes thanos-io#8446. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> (cherry picked from commit d54b40c)
The current implementation prunes tenants first and then updates `t.tenants`. TSDB clients are gathered from the `t.tenants` map. If a Series() call comes in the middle then the Querier could get an error like the following: ``` open chunk reader: can't read from a closed head ``` This commit tries to fix that by adding a "readOnly" mode where just before closing we now turn it on. It ensures that all RecvMsg() calls just return io.EOF. Also, did various tuning to tests so that they would pass more reliably. Now the first request doesn't fail. Adjust for that. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> (cherry picked from commit a7f04c2)
Deleting one by one is REALLY slow if you have thousands of blocks to delete. Use some concurrency by default. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> (cherry picked from commit bbe9710)
Signed-off-by: adezxc <adam.jasinski@vinted.com> (cherry picked from commit d05e90a)
…ed due to corrupt file or other errors (thanos-io#8480) Co-authored-by: Ben Ye <benye@amazon.com> (cherry picked from commit 094aa1d)
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> (cherry picked from commit b9d06d4)
Signed-off-by: Michael Hoffmann <mhoffmann@cloudflare.com> (cherry picked from commit d52bc93)
3108617 to
820edcf
Compare
Implement batching of series responses in gRPC to reduce overhead. Based on upstream commits 44c0fda and 49dde50. Changes: - Add batchable server wrapper that batches series into single messages - Handle batch responses in lazy and eager resp sets - Add ResponseBatchSize field propagation through proxy - Add flush chaining in flushable servers - Add unit tests for batch response handling Also fixes pre-existing build issues from previous cherry-picks: - Fix wg.Go() usage (Go 1.22+ only) in fetcher.go, blocks_cleaner.go, inprocess_test.go - Fix ulid.Bytes() -> ulid[:] for ulid v1 compatibility - Fix duplicate slices import in acceptance_test.go - Regenerate protobuf files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
820edcf to
2dd2481
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
Verification